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The Use of Readability Formulas in Patient 


Education Materials 
(abstract ) 

In light of recent trends for early discharge and subsequent increases 
in home care patient populations, the need for adequate patient 
education has become apparent. A major source of such patient 
education is through the use of printed patient cducation material 
(PEM). The adequacy of such materials has seldom been addressed. 
Among the measures of adequacy is the ease of comprehension. A 
major factor in determining comprehensibility is that of the reading 
difficulty level of the text. It is generally recognized that a grade 
level score or index of 6.5 to 8.5 is readily comprehended by the 
majority of adult Americans. Several formulae exist which have been 
shown to be reliable predictors of reading grade level. The current 
study applies the SMOG Index and Standal's procedure to PEMs gleaned 
from voluntary agencies, corporate sources, and three major teaching 
hospitals in the Kansas City area. 
Analysis of 10 representative PEMs revealed a mean SMOG Index score 
of 12.38 (+/—1.05) for the material analyzed. Adjusting the formula 
on the assumption that patient educators would define technical 
vocabulary terms resulted in a decrease in the SMOG Index to grade 
level 11.23 (+/—0.99). 
The authors conclude that the SMOG Index is a useful tool in 
determining the appropriateness of printed PEM. We also concluded 
that the reading grade level of PEMs may be beyond the skill level 


ability of the majority of our potential clientele. The authors 


suggest that further research include a broader based sample. 
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Introduction 


Since the advent of Diagnostically Related Groups (DRGs) and 
Prospective Payment Systems (PPSs) there appears to be a net decrease 
in the Average length of Stay (ALOS) during the hospitalization of 
COPD patients. Tyler has shown that for those DRG classifications 
(87, 95, 96, 97, 98) representative of COPD patients, there has been 
a mean decrease of 1.2 patient days.! One effect of the more rapid 
discharge of these patients; and of the reimbursement structure, is 
an increase in the numbers and a decrease in the relative "wellness" 
of patients receiving or needing home care. (e.g. Patients are being 
discharged sooner and in a less stable condition). Mathews reports, 
based upon Veterans Administration and U.S. Census Bureau data and 
subsequent projections that; "even using conservative estimates 
there may well be over 300,000 actual or potential users of home 
oxygen in the U.S."2 It is probable that, given the aging of the 
population, these figures will see a dramatic increase in the coming 
decade and will remain relatively high for the rest of the century. 

Given the projected increase in the home care population the 
need tor patient education in relation to their disease and the 
treatment of their disease will also increase, Current practice in 
this area relies heavily upon the use of patient education pamphlets 
and booklets. It seems apparent that the usefulness and efficacy of 
such material is to a large measure dependent on how well the patient, 
his family, and his friends and associates assimilate, understand, and 
put into action the content of such materials. A significant facvtor 


in the assimilataion, understanding and implementation of the 


information contained within the printed materials is the readability 
of the document. 

In order to determine the readability of currently available 
patient education materials, a study of a sample of such documents 
was designed. The question: "How casy to read are patient education 
materials?''; was approached through the research hypothesis which 
stated that the reading grade level of patient education documents 
will exceed the reading level ability of the average American citizen. 
A second hypothesis addressed the question; 'Will use of a modified 
scoring system which accounts for vocabulary teaching result in a 
significantly lower reading grade level score?'' The hypothesis stated 
that there would be a significant reduction in the reading grade level 
scores with the modified version as compared to the standard version 
of the system used. 

Review of the Literature 

Dale and Chall define readability as "... the sum total of all 
those elements within a given piece of printed material that effect 
the success that a group of readers will have with it. Among the 
elements which affect the readability of a document are length use of 
specialized terminology (jargon), sentence structure and use of 
polysyllabic words. Other factors which affect the readability of 
such materials are patient centered factors such as age, cognitive 
ability, education level, and in the case of the COPD patient, the 
degree of hypoxemia/hypercapnia induced cerebral dysfunction. 

All of the above factors interact to a greater or lesser degree 


in determining the ability of a given individual to read and 
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understand materiaits presented to him by patient educators. 
generally recognized that the American public, on the average, reads 


at a 6.5 to 8.5 grade level. Christ and Pharr in discussing the 


readability of informational brochures pose the following quest ion 
").. at what level should information brochures be written? ... some 
research seems Lo suggest that ... it would be reasonable to use a 
7th-8th grade readability level as a standard."4 Sinith maintains that 
"Census information shows that the mean educational level for persons 


25 and older is 12.3 years. Research, also, indicates that about half 


of the 12th graders read on or above the 12th grade level while about 
87% of them read above an 8th grade level." Monteith citing Fishers 
work states that 37% of the adult population "reads below an equivalent 
of 8.0 on standardized tests." 

Several types of ‘readability' indices have been developed. As 
with any instrument readability formulas do have their ] imitations; 
however, if used in the context for which they are meant, e.g., as a 


means of predicting difficulty, the information they yield is useful. 


The scores produced as a result of using these instruments are 


inversely related. The higher the score the lower the readability of 


the document. Conversely, the lower the score the less difficult and 


more readable is the document. 


It is important to begin with a clear picture of what readability 


formulas can and can not do. What they can do is predict a level of 


reading difficulty based on an analysis of sentence and word length in 


a given passage. What they can not do is to take into account the 


intraperson variables listed above or the actual content of the 


wm 


materials. dJittleman has pointed out that ''... word trequency and 
sentence length do not stand in simple relationship to reading 
difficulty seat? 

In other words, the level of difficulty is correlated to the 
length of words and sentences, however, other factors must also be 
considered, Arguments have been made that high scoring materials 
should be rewritten to produce lower scoring materials. ‘The very 
nature of PEMs raises the scores and attempts to rewrite them may 
result in gaps which the patient may have to fill in themselves. 
Although the patient may have less difficulty reading the PEM, his 
understanding of the material, when related ‘to the purpose of the PEM, 
may well be reduced. One possible method of reducing the score without 
rewriting the text was suggested by Standal.8 He proposed making an 
adjustment in the formula for vocabulary. ‘The educator would identify 
words whose caning he felt ought to be taught to the reader as 
vocabulary words, The reader would then be taught or presented with 
these words in a vocabulary format as an introduction to the PEM. 
These words, usually greater than two syllables, are then counted as 
only having one syllable and are therefore excluded from the 
calculation, Standal's procedure supposedly lowers the score, making 
the text more readable. 

For the purpose of this study an instrument which was simple to 
apply and which gave a score indicative of the documents grade level 
readability was selected. This instrument is called the SMOG Index. 
This instrument was developed in 1969 by McLaugh] in? and is highly 


correlated (rho 0.985, standard error of estimate 1.5159 grade 


levels) to the standard McCall-Crabbs criterion, which has been the 


benchmark against which all readability formulae have been measured 
since 1925.10) Interinstrument correlations between the SMOG Index 
and the other two most widely used scoring systems (Fry and Dale-Chall) 
is , also, extremely high with Spearman Rank Correlations at p = 0.001 
of rho = 0.87 for Dale-Chall and SMOG, and rho = 0.80 for Fry and 
smog, 11 

[t should be noted that both the Fry and Dale-Chall measures are 
predicated on a prediction criteria of 50-75% comprehension by the 
reader, while the SMOG measure is predicated on a 90-100% comprehension. 
Of further interest is the fact that the SMOG Index is consistantly 
two grades higher than either the Fry or the Dale-Chall1.9+ 10, 11 The 
differences between the 50-75% criterion of the Fry and Dale-Chall and 


the 100% comprehension of the SMOG Index may account for this discrepancy. 


Method 

Thirty patient education pamphlets dealing with respiratory 
diseases were obtained from voluntary health agencies, commercial 
organizations and three teaching hospitals in the Kansas City area. 
Ten of these were chosen as representative samples. This was 
accomplished by numbering the PEMs from one to thirty and then using 
a random number generator to select ten numbers from one to thirty. 
The numbers selected indicated the PEMs to be analyzed. 

In order to determine the readability of the sample documents the 
SMOG Index was chosen as the analysis tool. This instrument purports 


to measure the ‘readability index' in terms of prade level of reading 


ability. The analysis procedure consists of the following steps: 


1. Take thirty sentences, ten each from the beginning, 

middle, and end of the document. 

2. Exclude all proper names. 

3. Count the number of words with morc than two syllables. 

4, Take the square root of the number resulting from the count. 

5. Add 3 to the square root. 

6. The resultant number is the grade level at which the text 

can be read. 

As a mathematical expression the procedure appears as shown in 
figure 1. 

To provide a measure of comparison of the reading difficulty 
level, figures 2 and 3 compare the reading difficulty at various 
stages of education and the difficulty of some commonly used documents. 

It should be noted that the SMOG Index score of a given document 
is dependent on several factors in addition to the length of the words 
used. The length of the sentences will have a direct impact on the 
score. When one stops to think about it, it becomes obvious that the 
longer the sentence, the more opportunity there is to use long complex 
words and the more individual words the sample passages will contain, 
‘The use of technical terms, phrases, acronyms, and abbreviations also 
will tend to obscure meaning for the ininitiated. 

Stutistical Methods 

Each of the ten sample documents were analyzed for readability 

using the SMOG Index as an analysis tool. Simple descriptive 


statistics were calculated from the prouped scores. Additional 


analysis was performed using Standal's procedure and simple 


descriptive statistics were again calculated. The raw score data for 
both groups were subjected to analysis by t-tests for grouped data 
and by a Z-test analysis using the Basic Business Software "Statistics" 
(Apple If version). This was done for both the raw data and for ‘index 
scores using the national average reading level (7.5) as the population 
mean, 

Results 

Calculation of the simple descriptive statistics resulted in the 
determinations which are presented in figure 4. Additionally a 
1-tailed student's t-test for paired observations was performed on 
both the raw score data and the index score data to test the hypothesis 
that the Standal's procedure would result in a significant reduction 
of the reading difficulty level. This hypothesis was upheld as is 
shown by the results of the t-tests presented in figure 4 (Raw score 
t = 8,343, p< = 0.005, dF = 9). 

Conversion of the raw score data to grade level index data 
produced the results shown it. figure 5. Again, the results of the 
t-test for paired observations showed significant differences between 
the methods (t 7.307, p <= 0.005, df = 9). 

Analysis of the sample vs population means was accomp] ished 
using a z-test to determine the differences between the means for 
both the SMOG [Index and Standal's procedure as measured against the 
population mean of grade level 7.5. The results of this analysis as 
shown in figure 6 indicate that there is a significant difference 


between the rated readability of the sample documents and the abil ity 
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level of the general population. 


Discuss ion 

A concrete example of a new technique is often desirable. As 
such an example, the following passage from the excellent article by 
Feery, Weller, and Solliday, in the May, 1985 issue of Respiratory 
Carel2 has been reproduced and analyzed counting not only polysyllabic 
words, but also, technical jargon. (all indicated by under] ines). 
The authors hope that this example will clarify both the use of the 
SMOG Index and Standal's procedure and illustrate the potential benefits 
to be gained through their adaptation, when producing PEMs. 

"It has been common practice to test the respiratory drive of 

a comatose patient by disconnecting him from the ventilator 

and observing him for spontancous ventilatory efforts. Our 

experience argues against this practice. Five (22%) of our 

patients (Patients 4, 13, 14, 17, and 23) had a potentially 

dangerous fall in PaO2 and Patient 23 also developed an 

arrhythmia during apnea testing. Furthermore, if the 

clinical search for spontaneous respiratory efforts is not 

aided by instruments to continuously measure and record these 

efforts, then very small attempts at breathing may be missed 

by the observer, or gross movements duc to spinal reflexes 

may be mistaken for ventilation. 

Qur procedure is an improvement over carlier attempts at 

safer and more effective apnea testing. The exact apparatus 

and protocol for apnea testing may differ among institutions, 


but there are several points that we believe to be important. 
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First, diffusion respiration is effective in naintaining 

adequate PaO2 in the majority of nonbreathing patients 

providing there has been adequate denitrogenation. Second, 

the application of CPAP further increases the likelihood 

that PaO2 will be maintained. ‘Third, because the goal of 

apnea testing is to allow PaCo2 to rise to a level at which 

it is a strong ventilatory stimulus, frequent monitoring of 

PaCO2 is necessary to recognize when this point is reached 

so as not to discontinue a test prematurely or prolong it 

needlessly. The wide variation among patients in the rate 

of rise of PaCO2 makes it difficult to recommend one duration 

of testing for all." 

A count of the underlined items in the above passage reveals a 
total of 65 underlined items. If we accept that the remaining two 
sauples of ten sentence each needed to obtain a SMOG Index from, this 
paper would have relatively the same the SMOG Index for this paper 
would be in the area of 17.0 (This is not an indictment of this 
paper, the score seems appropriate for it's intended audience).  How- 
ever, based solely on the readability score, this paper would clearly 
be inappropriate for the lay audience. Use of the Standal procedure 
would lower this score, but net appreciably.  Exéuuples la, lb, and Ic 
illustrate the analysis by both methods and indicate the reduction in 
grade level by use of Standal's procedure. The SMOG Index score of 
16.95 obtained by analysis of the sample indicated a required reading 
level equivalent to that of a first year graduate student. Application 


of Standal's procedure (modified by inclusion of technical terms in 
if 
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the screening process as discussed above) resulted in a reduction of 
the reading level index to 14.22 This is equivalent to the reading 
level of a second year college student. While the reduction of 2.73 
grade levels appears to be satisfactory for the intended group, the 
ability of the material to be easily read and comprehended by a lay 
audience would not be greatly enhanced. 
Conclusions 

The ability to provide the patient with educational materials 
relating to the patient's disease, methods of treatment, and techniques 
designed to minimize the effects of the disease and: it's symptoms are 
an important facet of the rehabilitation process. The value of such 
materials is directly related to the patient's ability to read and 
understand the materials presented. The current study has shown that, 
for the limited sample analyzed, a significant portion of the target 
population may be at a significant disadvantage in terms of the ability 
to adequately read and comprehend patient materials now available. 

Use of readability formulae, such as the SMOG Index, allows the 
patient educator to determine the suitability of materials for his 
or her patient population. These formulae, in conjunction with 
vocabulary training, can permit the educator to 'fine tune' the 
teaching documents to the patients level of comprehension. Inclusion 
of a vocabulary section in patient education materials would seem to 
he highly desirable. None of the materials reviewed in this study had 
a readability index of less than 2.92 grade levels higher than the 
average reading level of the U.S. population when the SMOG index was 


calculated. Application on Standal's procedure had the effect or 
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lowering the gap between the population average and the lowest 
measured index score to 1.98 grade levels. ‘This was a signif icant 
reduction, but stil] resulted in a potential comprehension deficit. 

The authors conclude that readability formulas have a potential 

role in evaluating patient education materials. We do, however, 
caution that this potential may be limited to serving as a guideline 
as not all patients will have the same reading ability. We, therefore, 
suggest that questions regarding the patient's (and family's) 
educational attainment be incorporated as part of the patient 
evaluation process. We further conclude that the use of vocabulary 
training and the inclusion of a vocabulary at the beginning of each 
document will result in a more readable and therefore more valuable 
document . 

We would like to leave the reader with a final thought quoted 

from Lorge: 

"Readability formulae are no panacea. They do not tell 
anything about the ideas expressed or the interrelationships 
among them, At best, they are yardsticks. If they are 
not inflated into a recipe for writing, they are useful 
adjunct in the evaluation of written and spoken materials. 
Their use, however, cannot be a complete substitute for the 


wisdom of exper ience."13 
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SMOG INDEX = 3 + V (Wa + Wb + Wc 


Where: Wa = # of words ¥ two syllables in 1st ten lines. 
Wb = # of words >two syllables in middle ten lines. 
Wc = # of words ?two syllables in last ten lines. 


Figure 1: SMOG INDEX FORMULA 
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EDUCATION INDEX 


College 13 - 16 
High School 9 - 12 
Sixth Grade 6 
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EXAMPLES 


SOURCE SMOG INDEX 
Federal Register 20.95 
Respiratory Care 14.70 
The Atlantic 13.95 
People 10.87 
Newsweek 10.58 
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Figure 3: SMOG INDEX SCORES OF SELECTED PUBLICATIONS 
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READABILITY SCORES 


SPECIMEN: ''The Use of Apneic - Diffusion Respiration in 
the Diagnosis of Brain Death" * 


SMOG (Standard): 


WA = 65 (analyzed) 
WB = 65 (assumed) 
WC = 65 (assumed) 


SMOG INDEX = 3 + VWA + WB + WC 
SI = 3+ V65 + 65 + 65 
+ : 

+ 


SI = 3 + VT95 
SI = 3 + 13.96 
SI = 16.95 


*Feery, J.J.3 Waller, G.A.3 Solliday, N.; Respiratory Care: 
V 30, N 5, May 1985, p. 328-333. 


Example la: SMOG CALCULATION OF SPECIMEN DOCUMENT 
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READABILITY SCORES 


SPECIMEN: ''The Use of Apneic - Diffusion Respiration in 
the Diagnosis of Brain Death" * 


SMOG (STANDAL's PROCEDURE) : 
WA = 42 (analyzed) 
WB = 42 (assumed) 


WC = 42 (assumed) 
SMOG INDEX = 3 + VWA + WB + WC 
— Sle 3 eV 474 as + 42 
Sle Se Vr 
Sls 3.4 112 
Sl] 14.22 


* Feery, J.J.3; Waller, G.A.; Solliday, N.; Respiratory Care; 
V. 30, N.5, May 1985, p. 328-333. 
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Example 1b: MODIFIED SMOG CALCULATION OF SPECIMEN DOCUMENT 
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READABILITY SCORES 


SPECIMEN: ''The Use of Apneic - Diffusion Respiration in 
the Diagnosis of Brain Death" * 


Standard SMOG Index 16.95 
Standal's Procedure 14.22 
Change in grade level Zetd 


* Eeery, J.J.3 Waller, G.A.3 Solliday, N.3 Respiratory Care; 
V 30, N5s; May 1985; p. 328-333. 
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Example 1c: EFFECT OF STANDAL's PROCEDURE ON SPECIMEN DOCUMENT 


aa 


N 10 
MEAN i 67.8 
RANGE f 42-100 
SAMPLE VAR. 5. 304.40 
SD SAMPLE 

POP. VAR. 

SD POP, 

STD. ERR. 

t-statistic 8.343 

t-prob. 0.005* 


Figure 4: SIMPLE STATISTICS (RAW SCORES) 


SMOG STANDAL 


N 10 10 

MEAN 12.38 11.23 
RANGE 10.42-13.68 9, 48-1300 
SAMPLE VAR. 1.103 0.977 

SD. SAMPLE 1.05 0.987 

POP VAR. 0.993 0.876 

SE MEAN Uisoe 0.312 
t-statistic 7.307 

t-prob. €=0.005* 
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SMOG 
df 9) 
Z-score 15.43* 
< 0.001 


Figure 6: Z-TEST SCORE TABLE 


CONCLUSIONS 


1. A large proportion of the population may be at a disadvantage 
when required to read and comprehend PEM's. 


2. Use of readability formulae, especially in conjunction with 
vocabulary reviews, may decrease the problems associated with 
reading in this portion of the population, 


3. Patient assessment interviews should include assessment of the 
patients education status. 


4. Patient education materials should contain a vocabulary list 
at the beginning of each document. 


Figure 7: CONCLUSIONS 
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RECOMMENDATIONS 
Further research in this area should include studies of: 
1. Larger numbers of specimen documents. 
2. The effects of vocabulary vs no vocabulary 


training on the understanding of PEM's by 
patients. 


Figure 8: RECOMMENDATIONS VOR FURTHER STUDIES 
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"NOTHING IN THIS ACT SHALL BE CONSTRUED AS PRO- 
HIBITING OR PREVENTING AN INDIVIDUAL LICENSED UNDER 
OTHER LAWS FROM PERFORMING DUTIES OR FUNCTIONS COVERED 
UNDER THEIR SCOPE OF PRACTICE FOR WHICH THEY HAVE 
ADEQUATE TRAINING AND KNOWLEDGE ,"' 
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"IN CASES WHERE HARM MAY COME TO THE PATIENT WITH- 
OUT IMMEDIATE ACTION, AND WHERE A PHYSICIAN IS NOT 
AVALLABLE ‘THE PERSON LICENSED UNDER ‘TITS ACT MAY INSTI- 
TUTE THOSE FUNCTIONS HEREIN DEFINED, AT THE MINIMUM 
LEVEL NECESSARY TO PREVENT HARM, WITHOUT THE ORDER OF A 
PHYSICIAN," 
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